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I .  INTRODUCTION 

As  the  computing  needs  of  an  organization  expand,  the 
decision  to  add  to,  or  replace,  existing  computer  systems  is 
important  and  may  have  long  term  effects  on  an  organization. 
The  impact  can  be  financial,  operational  or  both.   The 
choice  of  computer  systems,  hardware  and  software  will  be  a 
major  factor  in  defining  the  information  processing 
capcdDilities  of  an  organization.   The  methods  for  selecting 
computer  systems  are  varied  and  have  evolved  over  the  past 
20  years. 

Computer  systems  have  become  less  expensive  and,  at  the 
same  time,  significantly  more  powerful.   Advances  in  data 
communication  technology  have  led  to  the  increased  use  of 
distributed  data  processing  (DDP)  and  decentralized  computer 
systems.  In  order  to  effectively  use  DDP  and  decentralized 
computing,  computer  systems  throughout  an  organization  must 
be  compatible  with  each  other.   Data  and  files  have  to  be 
transferable  and  usesdDle  on  all  systems  throughout  an 
organization . 

Once  a  decision  is  made  to  acquire  a  new  computer 
system,  an  organization  needs  to  ensure  that  its  selection 
is  the  best  choice  to  meet  its  present  and  future  computing 
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needs.   If  the  selected  computer  system  does  not  fulfill  the 
organizational  requirements  there  might  not  be  another 
chance  to  make  a  better  choice  due  to  cost  and  time 
constraints . 

The  emphasis  of  this  thesis  is  on  the  evaluation  and 
selection  phases  of  a  computer  system  acquisition  process. 
The  actual  selection  of  a  computer  system  is  preceded  by 
several  procedural  steps.   The  following  is  a  generalization 
of  those  procedures : 

•  Analyzing  the  requirements  and  computing  needs  of  the 
organization. 

•  Determining  and  defining  the  requirements  for  the 
computer  system. 

•  Sending  the  request  for  proposal  (RFP)  to  qualified 
vendors . 

•  Screening,  evaluating,  validating  and  comparing  the 
proposals . 

•  Selecting  the  computer  system. 

From  the  multitude  of  computer  systems,  operating 
systems,  software  and  peripherals,  a  user  has  to  choose 
those  products  that  best  meet  an  organization's  needs.   The 
difficult  task  facing  those  responsible  for  the  selection  is 
to  determine  by  what  standards  competing  systems  are 
compared  and  evaluated. 


Traditional  procedures  have  addressed  the  selection  of 
individual  computer  systems.   With  DDP  and  decentralized 
computing  becoming  more  prevalent,  there  is  a  need  to  apply 
new  methods  for  selection.   The  concern  is  no  longer  the 
selection  of  a  single  system;  it  is  the  selection  of  a  group 
of  systems  that  work  together  and  eneible  the  sharing  of 
files  and  data. 


II.  COMPUTER  SELECTION  PROCEDURES 

As  long  as  there  has  been  more  than  one  computer  system 
available  to  meet  a  user's  requirements,  there  has  been  a 
need  to  select  the  one  that  best  fits  the  job.   The  problem 
is  how  to  make  that  selection.   There  have  been  a  ntimber  of 
models  and  methods  proposed  and  used  to  select  computer 
systems.   The  various  methods  used  have  concentrated  on  the 
individual  stages  in  the  selection  process  and  on  specific 
techniques  for  each  stage. 

The  selection  process  consists  of  several  stages: 
analysis  of  needs,  determination  of  requirements,  request 
for  proposals,  evaluation  of  proposals  and  selection  of  a 
system.   The  first  two  steps  are  preliminary  to  the 
evaluation  of  the  proposals,  while  the  latter  two  are  often 
blurred  into  a  single  step:  evaluate  the  candidates  and  pick 
the  best . 

The  techniques  for  the  evaluation  of  computer  systems 
generally  are  either  simple  or  sophisticated.   The  simple 
methodologies  are  basically  intuitive,  unsupported  by  any 
theory  or  lacking  a  systemized  approach.   The  sophisticated 
methods  employ  an  analysis  of  vendors'  proposals  with  a 
ranking  of  the  proposed  computer  systems.   Following  this 


ranking,  a  hardware  performance  and  capability  analysis  is 
performed  on  the  most  desirable  alternatives.   If  the  system 
that  was  ranked  first  fails  to  perform  adequately  then  that 
system  is  rejected  and  analysis  is  carried  out  on  the  next 
highest  ranked  system. 

The  most  commonly  used  computer  system  evaluation 
methods  are : 

Weighted  Scoring.  This  commonly  used  approach  attempts  to 
overcome  evaluation  problems  by  combining  objectivity  with 
consideration  of  apparently  non-quantif icible  factors. 
Relative  weights  are  pre-assigned  to  all  system  items 
considered  important.   Each  competent  system  is  subjectively 
rated  with  respect  to  each  selected  system  item  and  then  the 
overall  score  is  computed  for  each  vendor.   The  rating  is 
subjective:  from  "no  good"  to  "entirely  adequate".   The 
system  with  the  highest  total  score  is  considered  most 
desiraJDle.  (Borovits  and  Zviran,  1987) 

The  advantage  of  this  method  is  that  it  is  simple.   It 
also  enables  one  to  perform  sensitivity  analysis  of  the 
results  for  changes  in  the  importance  weights  of  the 
attributes.   Its  disadvantage  is  that  it  is  not  normative, 
i.e.  it  is  not  based  on  a  system  of  axioms  expressing 
rational  behavior  rules  expected  of  a  decision  maker.   Thus, 


it  does  not  require  an  examination  of  assumptions  of 
independence  in  the  attributes  and  the  absence  of  such  an 
examination  may  result  in  deviations.  (Shoval  and  Lugasi, 
1987)   The  following  example  demonstrates  this: 

Two  proposals,  A  and  B,  are  exsimined  according  to  two 
independent  attributes,  "vendor  support"  and  "hardware 
performance".   If  A  and  B  receive  the  same  score  in  the 
evaluation  and  if  "vendor  support"  is  evaluated  as  entirely 
adequate  in  proposal  A  and  no  good  in  proposal  B  while 
"hardware  performance"  is  entirely  adequate  in  B  and  no  good 
in  A,  then  the  "hardware  performance"  of  B  may  never  be 
available  because  of  its  no  good  "vendor  support". 
(Timmreck,  1973) 

This  method  does  not  allow  for  an  examination  of 
consistency  of  the  evaluators.   Due  to  its  nature,  the 
method  does  not  consider  risk  and  uncertainty.  It  should  be 
noted  that  granting  scores  in  every  attribute  is  in  itself  a 
difficult  task  and  influenced  by  subjective  considerations. 
(Shoval  and  Lugasi,  1987) 

Multi -Attribute  Utility  Model.   Shoval  and  Lugasi  (1987)  use 
of  Keeny' s  multi-attribute  utility  model  allows  an 
evaluation  of  the  utility  function  of  the  attributes  and  the 


calculations  of  their  weights .   It  differs  from  other  models 
in  that  it  considers  risk  and  uncertainty. 

The  application  of  the  model  requires  both  utility  and 
preference  independence.   Utility  independence  claims  that 
for  attributes  XI... Xn  the  utility  of  attribute  Xi  does  not 
depend  on  the  remaining  attributes .   Preference  independence 
claims  that  preference  between  every  pair  of  attributes  does 
not  depend  on  the  fixed  level  of  remaining  attributes . 

There  are  two  variants,  which  differ  in  the  way  they 
refer  to  risk,  the  additive  model  and  the  multiplicative 
model.   The  application  of  either  depends  on  a  decision 
maker's  attitude  towards  risk. 

In  order  to  evaluate  the  utility  function  of  the 
attributes  and  to  calculate  the  weights/constants  of  the 
attributes,  a  gambling  technique  is  used,  a  utility  function 
of  the  Von-Neuman-Morgenstern  type. 

Evaluation  of  the  decision  maker's  utility  function  is 
based  on  the  axioms  of  transitivity  and  continuity.   If 
these  axioms  are  satisfied,  it  is  possible  to  form  a  utility 
function  for  the  decision  maker  for  each  attribute. 

The  multi-attribute  utility  model  is  normative,  since  it 
forces  a  decision  maker  to  accept  a  set  of  axioms  which 
express  preferences  and  it  also  requires  an  examination  of 


independence  assumptions  before  the  model  is  applied.   The 
model  also  enables  one  to  make  sensitivity  analyses. 
Compared  with  other  models  this  model  is  more  difficult  for 
the  decision  maker  to  understand  and  apply,  since 
determining  a  probability  at  which  a  decision  maker  is 
indifferent  between  alternatives  is  not  always  clear  and 
acceptable.  (Shoval  and  Lugasi,  1987) 

Cost -Effectiveness  Ratio.   This  is  simply  a  sub-category  of 
the  weighted  scoring  technique,  where  the  cost  of  the  entire 
computer  system  is  divided  by  the  sum  of  its  points.   Th.e 
system  having  the  highest  total  score  is  considered  the  most 
desirable.  (Borovits  and  Zviran,  1987) 

Cost -Value.   This  approach  uses  cost  as  the  sole  measure. 
Cost  and  benefits  are  assigned  to  the  desirable  features  of 
each  proposed  system.   The  sum  of  these  values  is  then 
subtracted  from  the  total  cost  of  the  system.   The  result 
represents  the  cost  of  the  system  to  meet  to  meet  the 
mandatory  requirements.   The  system  with  the  lowest  cost  is 
the  most  desirable.  (Timmreck,  1973) 

Requirement -Costing  Technique.   Here  the  estimated  cost- 
savings  benefits  are  assigned  to  a  set  of  desirable 
features .   Those  not  part  of  the  proposed  system  are  then 
added  to  the  total  cost  of  the  system.   In  addition,  if  a 


desirable  feature  is  offered  at  an  additional  cost  that  is 
less  than  the  estimated  benefit,  the  incremental  cost  is 
added  to  the  system  cost .   The  system  with  the  lowest 
resulting  cost  is  considered  the  most  desirable. 
(Borovits,  1984) 

Dynamic  i^>proach.   This  approach  employs  a  projected  work 
load  growth  trend  and  cost-effectiveness  ratios  of  each 
vendor's  proposed  product  line.   The  underlying  assumption 
is  that  it  would  be  feasible  to  install  a  system  for  as 
little  as  one  year,  but  only  if  it  will  be  replaced  by  a 
compatible  system  from  the  same  manufacturer' s  prod_uct  line 
and  replacement  accords  with  the  dictates  of  work  load. 
Using  the  projected  work  load  trend  and  measurement  of 
system  capacity,  a  schedule  of  replacement  is  worked  out. 
The  cost  of  each  system  (as  determined  by  one  of  the  methods 
described  above)  over  the  period  of  its  employment  is  then 
discounted  with  respect  to  an  estimate  of  cost  of  capital 
and  its  expected  future  life.   The  product  line  with  the 
lowest  discounted  present  cost  is  the  most  desirable.  (Ein- 
Dor,  1977) 

Present  Value  Analysis.   The  objective  here  is  to  determine 
which  system  will  cost  the  least  and  benefit  the 
organization  most .   The  net  present  value  of  the  proposed 


system  should  be  determined  by  discounting  all  cash  flows 
associated  with  it,  using  an  organization's  cost  of  capital 
as  the  discount  rate.   The  benefits  projected  by  the 
proposed  new  system  are  the  net  cash  inflows .   The  system 
with  highest  net  present  value  is  the  most  desirable. 
(Borovits  and  Zviran,  1987) 

Efficient -Frontier  Model.   In  this  model,  the  preferred 
system  is  that  which  is  dominant  in  all  the  attributes 
considered  according  to  a  decision  maker's  preference. 

This  method  enables  one  to  screen  out  alternatives  that 
are  inferior  in  all  attributes  or  are  identical  in  some  and 
inferior  in  ac  least  one.   The  alternatives  remaining  after 
this  screening  form  an  "efficient  frontier" .   There  remains 
the  problem  of  selecting  a  preferred  alternative;  this 
requires  other  methods.   Therefore,  this  model  is  not 
sufficient.   It  is  applicable,  at  most,  in  performing 
initial  screening.  (Shoval  and  Lugasi,  1987) 
Lexicographical  Ordering.   In  this  model,  alternatives  are 
rank-ordered  according  to  a  dominant  attribute,  the  most 
important  one,  e.g.  CPU  power.   Clearly,  this  is  possible 
only  if  a  dominant  attribute  exists  and  cannot  be  traded  off 
with  others.   This  model  can,  at  most,  help  in  the  initial 


screening  of  alternatives  which  are  deficient  in  an 
important  attribute  but  it  cannot  choose  the  best .  (Ahituv 
and  Nexomann,  1986) 


III.  COMPUTER-FAMILY  SELECTION 

The  selection  procedures  being  discussed  focus  on  the 

selection  of  a  group  of  systems  that  will  work  together, 

sharing  files  and  data.   This  group  of  systems  has  been 

called  a  computer-family  and  defined  as: 

Conqputars  of  the  same  type,  consisting  of  several  models 
from  the  same  manufacturer's  product  line,  ranging  from 
microcos^uter  to  mainframe,  with  full  compatibility  in 
the  operating  system  and  the  system's  software,  to 
enable  transfer  of  application  software  from  one  family 
member  to  another  without  change. 
(Borovits  and  Zviran,  1987) 

An  example  of  a  computer-family  is  Digital  Equipment 
Corporation's  (DEC)  VAX  8840  mainframe  computer,  VAX  6220 
minicomputer  and  Microvax  3600  super-microcomputer.   Another 
example  consists  of  a  Prime  6650  mainframe  computer,  a  Prime 
4450  minicomputer  and  a  Model  4050  super-microcomputer. 

Selecting  a  computer-family  is  a  more  complex  process 
than  selecting  a  computer  system.   The  process  will  have  a 
lasting  effect  on  an  organization  because  there  will  be  one 
computer-family  from  which  current  and  future  hardware 
acquisitions  will  be  made.   This  will  help  ensure  uniformity 
and  compatibility  in  information  processing  throughout  an 
organization . 


For  DDP  and  decentralized  computing  the  compatibility 
obtained  in  hardware  and  software  will  preclude  the  systems 
integration  problems  encountered  while  using  mismatched 
systems.   The  benefits  of  system-wide  compatibility  are 
exemplified  by  the  ability  to  transfer  application  software 
from  one  family  member  to  another  using  a  common  operating 
system. 

A  number  of  methods  are  used  for  computer  system 
selection.   Some  of  the  concepts  presented  in  the  existing 
methodologies  form  the  basis  for  a  methodology  to  deal  with 
computer-family  selection.   The  procedures  presented  here 
are  a  ten-step  generic  evaluation  and  selection  methodology. 
(Borovits  and  Zviran,  1987) 

Figure  1  portrays  a  work  flow  diagram  of  this  process. 
The  work  flow  diagrcim  highlights  the  processes  that  occur 
after  an  analysis  of  an  organization  takes  place  and 
systems'  requirements  document  are  developed. 

The  stages  in  this  process  are: 

Step  1.  Identification  of  possible  vendors  and 
manufacturers:  After  studying  the  requirements  document  it 
may  be  determined  which  manufactures  or  vendors  may  be  able 
to  provide  products  that  will  meet  an  organizational  needs. 


step  1 .  Identification  of  possible 
vendors  and  manufacturers . 


Step  2.  Primary  elimination  of 
irrelevant  candidates . 


Step  3 .  Determination  of 
mandatory  requirements . 


Step  4.  Examination  of  vendors'  compliance 
with  mandatory  requirements. 


Step  5.  Setting  quantitative  and  qualitative 
criteria  and  respective  weighting  scales. 

Step  6.  Writing  the  RFP  to  be  addressed 

to  selected  vendors. 
I 

♦ 

Step  7.  Receiving,  comparing  and  analyzing  bids. 
Step  8.  Concluding  final  list  of  vendors. 

\ 

Step  9 .  Performance  of  hardware 
and  software  benchmarks . 


I 


Step  10.  Drawing  final  conclusions 
and  selection  of  best  computer-family. 


Figure  1.  Selecting  a  computer-family:  a  work  flow 
diagram.  (Borovits  and  Zviran,  1987) 
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A  list  of  possible  vendors  can  be  generated  and  used  in 
future  stages. 

Step  2.  Preliminary  elimination  of  irrelevant 
candidates:   Further  study  of  an  initial  list  of 
manufacturers  and  vendors  should  reveal  apparently 
unsuitcQ^le  candidates .   They  should  be  pared  from  the 
original  list  to  produce  a  manageable  list  of  potential 
(relevant)  vendors.   Reasons  for  elimination  are  for 
unsuitcdDility  in  meeting  requirements  or  because  they  do  not 
have  a  complete  computer-family. 

Step  3.  Determination  of  mandatory  requirements:   From 
the  systems  requirement  document  and  from  organizational 
policy  certain  features  and  characteristics  of  a  computer- 
family  are  identified  as  prerequisites  for  further 
consideration.   These  are  criteria  that  must  be  met  by  any 
potential  computer-family. 

Step  4.  Excunination  of  vendor's  compliance  with 
mandatory  requirements:  On  the  basis  of  mandatory 
requirements,  a  questionnaire  is  formulated  and  addressed  to 
all  relevant  vendors.   Responses  to  this  questionnaire 
provide  a  selection  committee  with  basic  information  about 


proposed  computer-fcunilies  and  each  product  line.   The 
responses  are  screened  to  determine  if  they  meet  mandatory 
requirements.   The  vendors  remaining  after  this  elimination 
procedure  constitute  a  mailing  list  for  requests  for 
proposals  (RFP) . 

Step  5.  Setting  quantitative  and  qualitative  criteria 
and  respective  weighting-scales:  The  varicibility  of  the 
capabilities  and  performance  characteristics  of  each  element 

(hardware,  software,  service)  of  a  computer  system  is  very 
large,  and  the  number  of  permutations  is  considerable.   A 
properly  constituted  set  of  quantitative  and  qualitative 
criteria  is  critical  for  the  acquisition  of  a  satisfactory 
computer-family.   Furthermore,  vendors  are  expected  to 
propose  not  a  single  computer  system  but  rather  a  wide 
product  line  of  systems  working  together.   This  raises  a 
problem  of  evaluating  and  comparing  proposals.   Borovits  and 
Zviran  (1987)  suggested  that  the  quantitative  selection 
criteria  be  applied  to  at  least  the  following  five  issues, 
each  of  which  has  its  own  partial  list  of  guidelines. 

[Fig.  2] 
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HARDWARE    (for  each  model  within  the  family) 

*  memory  size  (minimum,  maximum,  units  of  expansion) 

*  disks  (number  of  drives,  minimum  and  maximum  capacity) 

*  tapes  (maximum  number  of  drives,  density) 

*  diskette  readers  (diskette  size,  density) 

*  terminals 

*  printers 

*  backup  features 

*  miscellaneous 

SOFTWARE    (for  the  entire  family) 

*  system  software  (uniformity  of  operating  system,  compilers, 
utilities,  etc.,  throughout  the  fsunily) 

*  programming  languages  (compilers  and  interperters,  options 
and  extensions,  adherence  to  standards) 

*  development  tools  (existence  of  application  generators,  re- 
port generators,  full  screen  editor,  debugging  aids,  macro 
facilities,  etc.) 

*  additional  software  (electronic-spreadsheet,  word-processor, 
electronic-mail,  DBMS,  etc.) 

*  ability  to  transfer  software  within  the  family  (source  mod- 
ules, load  modules,  procedures)  without  change  or  conver- 
sion 

COMMUNICATION 

*  communication  protocols 

*  existence  of  LAN,  possibilty  of  connection  to  LAN 

*  possibilty  to  transfer  files,  programs  and  procedures  through 
communication 

*  network  transfer  rate 
CONVERSION 

*  possibilty  to  convert  application  software  (programs  and 
procedures)  from  present  system  to  proposed  computer- 

f  aunily 
ENVIRONMENT 

*  environment  requirements  (such  as  airconditioning, temper- 
ature, humidity,  etc.) 


Figure  2.  List  of  criteria.  (Borovits  and  Zviran,  1987) 


Certainly  not  less  important,  though  non-quantifiable, 
are  qualitative  criteria  such  as: 

•  how  widespread  is  the  use  of  the  proposed  computer- 
fcunily 

•  vendor  support,  such  as  training,  maintenance  philosophy 
and  practice 

•  software  houses  specializing  in  the  proposed  computer- 
f  cunily 

•  current  users'  opinions  and  vendor's  reputation 
Once  a  detailed  and  complete  set  of  data  relating  to  all 
criteria  is  collected,  weights  are  assigned  to  all  items, 
both  qualitative  and  quantitative,  indicating  their  relative 
importance.   These  weights  are  fundamental  to  the  evaluation 
process,  explicitly  indicating  that  the  objective  of  the 
evaluation  process  is  to  ensure  the  selection  of  a  computer- 
family  that  best  fits  the  needs  (present  and  future)  of  an 
organization. 

Step  6.  Writing  the  RFP  to  be  addcessed  to  selected 
vendors:   The  primary  advertising  medium  is  the  RFP,  which 
is  issued  by  a  procuring  organization  and  contains  the 
system  requirements.   Typically,  the  RFP  will  include  a 
summary  list  of  specific  requirements  according  to  which  the 
vendors  will  be  asked  to  write  their  proposals,  describing 


how  they  will  meet  each  requirement.   The  RFP  should  consist 
of  two  major  parts,  the  first  relating  to  each  model  within 
the  computer-family  and  the  second  to  the  computer-family  as 
a  uniform  entity.   The  second  part  is  more  general  in  nature 
and  concentrates  upon  such  issues  as  system  software, 
conversion  of  present  application  software  to  the  new 
computer-family,  environment,  etc.   The  RFP  should  be 
addressed  to  those  vendors  satisfying  mandatory 
requirements. 

Step  7.  Receiving,  comparing  and  analyzing  bids:   The 
problem  of  analyzing  and  comparing  bids  is  particularly 
important,  since  it  is  more  than  possible  that  none  of  the 
computer-families  under  consideration  may  exhibit  a  clear 
dominance  over  others.   This  stage  is,  therefore, 
fundeunental  to  the  interim  evaluation  and  selection  process 
and  should  be  accorded  an  appropriate  measure  of  time  and 
effort.   It  is  required  that  the  bids  be  submitted  in 
writing  and  it  is  expected  that  they  will  conform  to  the 
style  indicated  in  the  RFP,  so  that  the  selection  process 
will  not  be  affected  by  style  of  expression  and  use  of 
selling  techniques . 


The  first  step  in  the  process  of  analyzing  bids  to 
subjectively  rate  each  competing  system  with  respect  to  each 
evaluation  criterion.   A  score  is  assigned  to  each  item. 
The  weighted  score  is  then  computed  for  each  proposed 
system.   The  next  stage  is  the  most  problematic  since  there 
are  no  methods  or  guidelines  for  comparing  computer- 
families.   The  proposed  method  consists  of  the  following 
steps : 

•  Five  categories  of  computers  are  established: 

microcomputers  (single  station) 

super -microcomputers 

minicomputers 

super -minicomputers 

mainframe  computers 
These  categories  represent  differences  in  computing  power 
and  other  major  hardware  characteristics . 

•  Relative  weights  are  assigned  to  each  category  according 
to  its  importance  for  an  organization. 

•  Each  model  within  each  computer-family  is  assigned  to  a 
categoify  on  the  basis  of  criteria  such  as  memory  size, 
maximum  niomber  of  terminals,  maximum  disk  capacity  and 
relative  performance. 

•  For  each  criterion,  a  computer  model,  or  models,  is 
selected  within  each  category,  which  has  dominance  over 
its  competitors  and  the  score  of  100  is  assigned  to  it. 
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All  other  models  of  the  remaining  computer-families  are 
then  respectively  an  subjectively  scored  on  a  scale  of 
0  -  100. 

•  The  total  score  is  calculated.   The  score  attained  for 
each  criterion  is  the  average  weight  calculated  for  all 
five  categories. 

•  A  comparative  table  is  drawn  up  [Fig.  3]  giving  the 
scores  attained  by  each  family  for  each  criterion. 

•  The  final  score  achieved  by  each  family  is  calculated. 


Step  8 .  Drawing  up  a  final  list  of  vendors :   On  the 
basis  of  the  final  scores  attained  by  each  computer-family  a 
selection  committee  is  able  to  disqualify  irrelevant 
computer-families,  and  select  up  to  three  or  four  vendors 
most  likely  to  succeed.   These  computer  families  are  then 
further  tested  to  ensure  they  have  the  proper  capaibilities 
and  characteristics. 


Figure  3 . 


Outline  of  comparative  table. 
(Borovits  and  Zviran,  1987) 


step  9.  Benchmarks  for  performance  of  hardware  and 
software:   Study  of  the  written  proposals  is  sufficient  for 
evaluating  fulfillment  of  certain  recjuirements .   In  such 
cases,  benchmarking  must  be  performed.   A  benchmark,  in  the 
context  of  this  discussion,  is  a  set  of  live  tests  designed 
to  examine  the  computer  systems  (hardware  and  software) 
proposed  in  response  to  an  RFP .   Selection  of  criteria  to  be 
tested  may  be  performed  according  to  their  importance  for  an 
organization  using  the  relative  weights  already  assigned. 
The  purpose  of  these  benchmarks  is  to  verify  a  system' s 
characteristics  before  drawing  final  conclusions. 

Step  10.  Final  conclusions  and  selection  of  the  best 
computer-family:   After  benchmarks  have  been  performed  and 
all  essential  characteristics  of  a  proposed  computer-family 
have  been  deemed  satisfactory,  a  selection  committee  will 
review  and  reconsider  the  relevant  scores  assigned  to  each 
competing  computer-family. 

Finally,  a  committee  will  pick  the  best  as  the  one 
recommended  to  be  an  organization's  computer  -  f  cimily  .   The 
recommendations  will  then  be  submitted  to  an  organization's 
management  for  approval  and  adoption.   A  problem  faced  by 
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those  involved  in  the  selection  process  is  how  to  compare 
criteria  and  how  to  prioritize  them  according  to  their 
importance  to  the  decision  making  process.   There  will  be  a 
large  number  of  criteria,  some  quantifiable  and  others  non- 
quantifiable,  whose  importance  to  the  selection  process  will 
be  compared  with  each  other. 
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IV.  THE  ANALYTICAL  HIERARCHY  PROCESS 

The  methodology  developed  by  Borovits  and  Zviran  (1987) 
for  computer -family  selection  discussed  briefly  the  process 
of  assigning  relative  weights  to  each  category  of  computer 
in  a  family  being  evaluated  (Step  5) .   The  process  continues 
with  each  criterion  assigned  a  relative  weight  so  that  it 
can  be  evaluated  and  scored.   In  a  case  study  by  Borovits 
and  Zviran  (1987) ,  the  assigning  of  the  relative  weight  and 
the  actual  scoring  of  computers-families  was  done 
subjectively. 

Using  subjective  weighting  and  scoring  can  reduce  the 
overall  effectiveness  of  the  process.   There  is  little 
precision  and  it  may  be  difficult  to  achieve  replication 
using  subjective  weighting  and  scoring. 

An  objective  weighting  and  scoring  system  can  improve 
this  process.   An  objective  weighting  system  should  be 
characterized  by  consistency  in  the  assignment  of  weights. 
Consistency  is  a  factor  that  can  be  computed  and  compared  to 
a  standard.   The  standard  is  somewhere  less  than  perfect 
consistency  and  greater  than  intolerable  inconsistency.   If 
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a  standard  is  exceeded,  this  indicates  randomness  in  the 
assignment  of  values.   Should  this  occur,  the  criteria 
should  be  re-evaluated.  (Saaty,  1980) 

An  objective  method  that  can  be  adapted  for  the 
comparison  of  a  large  number  of  objects,  as  is  the  case  in 
comparing  computer-families,  is  the  Analytical  Hierarchy 
Process  (AHP)  developed  by  Saaty  (1977) . 

The  AHP  can  properly  assess  the  importance  of  a  large 
number  of  interacting  factors,  develop  priorities  among  the 
factors  and  choose  a  best  alternative  (Saaty,  1980)  . 
Seidmann  and  Arbel  (1985)  suggested  an  application  of  the 
AHP  to  the  process  of  microcomputer  selection.   A  large 
number  of  or  attributes  were  used  to  compare  microcomputers . 
The  use  of  AHP  allows  the  determination  of  both  weights  and 
scores  for  each  attribute  for  each  alternative,  using 
matrices  to  perform  pairwise  comparisons  between 
alternatives.   Once  weights  and  scores  are  obtained,  the 
final  score  of  each  alternative  is  calculated  according  to  a 
weighted  scoring  technique. 

In  order  to  determine  the  weights  of  n  attributes,  a 
decision  maker  fills  up  a  matrix  with  dimensions  n  x  n   in 
which  a  pairwise  comparison  is  made  between  every  two 
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attributes.  (Fig.  4)   Thus,  in  each  cell  (i,j)  in  the 
matrix,  a  decision  maker  expresses  the  relative  importance 
of  attribute  i  with  respect  to  attribute  j.   (It  is  only 
necessary  to  fill  up  half  of  the  matrix  since  a  ^  .,  is  the 
reciprocal  of  a  ^^.)       Then,  an  eigenvector  of  the  matrix  is 
calculated  for  a  maximum  eigenvalue.   The  eigenvector  is 
normalized  so  that  the  total  sum  of  its  elements  is  1 .   The 
values  of  this  eigenvector  constitute  the  attribute  weights. 


Poilti-    Ideolog-        Techno-        Eigen- 
Economlc        Social  cal  ical  logical         vector 


1/6 
1/2 


Figure  4.  Matrix  of  attributes.  (Saaty,  1982) 

To  determine  the  attribute  scores  for  each  alternative, 
a  decision  maker  fills  a  matrix  of  pairwise  comparisons 
between  alternatives.   Altogether,  n  matrices  are  filled 
(one  for  each  attribute) .   The  dimension  of  every  matrix  is 
the  number  of  alternatives.   Here,  too,  a  nofinalized 
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eigenvector  is  calculated  and  the  elements  of  each  such 
vector  express  the  scores  of  the  alternative  for  each 
attribute. 

Pairwise  comparisons  are  accomplished  with  a  relative 
numerical  scale.   Saaty  (1977),  in  agreement  with  G.A. 
Miller's  1956  experiments  proving  individuals  cannot 
simultaneously  compare  more  than  seven  objects  (plus  or 
minus  two)  at  the  same  time  without  being  confused,  chose  a 
numerical  scale  of  1  -  9.   Using  this  scale,  1  expresses 
identity  between  two  objects  being  compared  and  9  expresses 
an  cUosolute  preference  of  one  over  the  other.  [Fig.  5] 

The  Eigenvector  method  enables  one  to  examine  a  decision 
maker's  consistency,  using  appropriate  measures  (in  essence, 
if  a  decision  maker  is  consistent  in  a  certain  matrix,  the 
maximum  eigenvector  of  the  matrix  equals  the  order  of  that 
matrix) .   On  the  other  hand,  this  method  ignores 
independence  between  attributes.   It  also  does  not  consider 
risk  and  uncertainty.   Nor  does  it  enaJDle  one  to  perform 
sensitivity  analyses.  (Shoval  and  Lugasi,  1987) 

The  AHP  removes  the  guess  work  from  the  assegament  of 
the  relative  importance  of  various  criteria.  The  AHP  can 
assist  in  analyzing  several  variables  simultaneously.   A 
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hierarchy  is  established  so  that  sub-criteria  relevant  to  a 
main  criteria  can  be  prioritized  and  comparisons  can  be  made 
of  minute  details.  (Saaty,  1980) 


Intansity   of 
Importance 


Explanation 


Essential  or  strong  Impor- 
tance of  one  element  over 
another 


Intermediate  values  between 
two  adjacent  judgments 

If  activity  i  has  one  of  the 
preceeding  numbers  assigned 
to  it  when  compared  with 
activity  j,    then  j   has  the 
reciprocal  value  when  com- 
pared with  1 


Two  elements  contribute 
equally  to  the  property 

Experience  and  judgment 
slighty  favor  one  element 
over  another 

Experience  and  judgment 
strongly  favor  one  element 
over  another 

An  element  is  strongly 
favored  and  its  dominance  is 
demonstrated  in  practice 

The  evidence  favoring  one 
element  over  another  is  of  the 
the  highest  possible  order  of 
affirmation 

Comprimise  is  needed 
between  two  judgments 


Figure  5.  Saaty' s  comparison  table.  (Saaty,  1982) 
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V.  PROPOSED  METHODOLOGY 

Applying  the  AHP  to  Step  5  of  Borovits  and  Zviran's 
methodology  (Fig.  1)  makes  the  resolution  of  ranking  and 
weighting  alternatives  less  arbitrary. 

In  selecting  a  computer-family,  specific  attributes, 
capabilities,  and  perfoirmance  characteristics,  collectively 
known  as  criteria,  are  used  as  the  basis  for  comparison 
between  the  competing  alternatives.   In  Step  5  of  Borovits 
and  Zviran' s  methodology,  criteria  are  identified, 
prioritized  and  weighted  according  to  their  significance  and 
value  in  the  selection  process.   This  procedure  yields  a 
maximum  value  for  each  criterion.   Using  the  maximum  values 
for  all  of  the  criteria,  the  highest  possible  score  for  each 
computer-family  is  computed.  The  maximum  attainable  scores 
in  Step  5  are  used  in  Step  7 . 

The  AHP  allows  a  decision  maker  to  objectively  create  a 
prioritized  and  weighted  list  of  criteria.   At  each  level  of 
the  hierarchy,  every  criterion  is  compared  to  all  the  others 
in  its  group,  on  a  one-to-one  basis.   Using  the  scale  and 
descriptions  from  Figure  5,  a  scoire  for  each  pairwise 
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comparison  is  obtained.   These  scores  are  inserted  into  a 
matrix  to  compute  the  weight  of  each  criterion  by  Saaty's 
eigenvalue  method.   The  weight  of  each  criterion  is  used 
with  the  weights  of  the  criteria  that  are  above  it  in  the 
hierarchy  to  compute  a  maximum  value. 

To  facilitate  the  incorporation  of  the  AHP  into  Step  5 
of  Borovits  and  Zviran' s  methodology  (Fig.  1),  setting 
quantitative  and  qualitative  criteria  and  respective 
weighting  scales,  the  prioritization  of  all  criteria  is 
realized  by  completing  six  sub-steps. 

The  sub  steps  are  listed  below  in  the  order  they  are  to 
be  accomplished. 

5.1.  -  Prioritize  the  overall  importance  of  qualitative 

versus  quantitative  criteria. 

5.2.  -  Select  applicable  computer  categories.   Each 

category  is  evaluated  separately. 

5.3.  -  Select  criteria  for  each  computer  category. 

5.4.  -  Select  sub-criteria  for  each  criterion  until 

there  are  no  more  sub-criteria. 

5.5.  -  Prioritize  and  weight  all  categories,  criteria, 

and  sub-criteria. 
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5.6.  -  Calculate  the  absolute  weights  for  all  criteria 

and  s\ib-criteria. 
In  Step  1,    the  process  of  receiving,  comparing  and 
analyzing  bids  is  comprised  of  two  sub-steps: 

7.1.  -  Assign  each  relevant  model  of  computer  from  a 

proposed  computer- family  to  a  category 
according  to  estciblished  criteria. 

7.2.  -  Evaluate  each  computer -fautiily  in  accordance  with 

the  criteria  estcJDlished  in  Step  5. 
The  advantage  in  applying  the  AHP  to  this  methodology  is 
that  greater  objectivity  is  achieved.   No  longer  will  an 
individual,  or  group,  have  to  compare  a  large  number  of 
criteria  and  rank  them,  including  the  assignment  of  a 
relative  value  for  each  object.   All  comparisons  are  done 
one  to  one.   The  ranking  and  weighting  of  the  criteria  is 
computable  and  replicable. 


step  1.   Identification  of  possible  vendors  and 
manufacturers . 

Step  2.   Primary  elimination  of  irrelevant  candidates. 

Step  3.   Determination  of  mandatory  requirements. 

Step  4.   Excimination  of  vendors'  compliance  with 
mandatory  requirements . 

Step  5.   Setting  quantitative  and  qualitative  criteria 
and  respective  scales. 


5.1.  Prioritize  overall  importance  of  qualitative 
and  quantitative  criteria. 

5.2.  Select  applicable  computer  categories.   Each 
category  is  evaluated  separately. 

5.3.  Select  criteria  for  each  computer  category 

5.4.  Select  sub-criteria  for  each  criterion  until 
there  are  no  more  sub-criteria. 

5.5.  Prioritize  and  weight  all  categories,  criteria 
and  sub-criteria. 

5.6.  Calculate  the  absolute  weights  for  all  criteria 
and  sub-criteria. 


Step  6.   Writing  the  RFP  to  be  addressed  to  selected 
vendors . 

Step  7.   Receiving,  comparing,  and  analyzing  bids. 


7.1.  Assign  each  relevant  model  of  computer  from  a 
proposed  computer-fcimily  to  a  category  according 
to  predetermined  criteria. 

7.2.  Evaluate  each  computer-family  in  accordance 
with  criteria  established  in  Step  5. 


Step  8 .   Drawing  up  a  final  list  of  vendors . 

Step  9 .   Performance  of  hardware  and  software  benchmarks . 
Step  10.  Drawing  final  conclusions  and  selection  of  best 


Figure  6.  Updated  workflow  diagram. 
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VI .  DEVELOPING  SELECTION  CRITERIA  AND  SCALE 

A.   SELECTING  AND  PRIORITIZING  CRITERIA 

In  order  to  select  a  computer- faunily  that  best  fulfills 
its  requirements,  an  organization  must  designate  the 
qualities  that  will  be  used  to  compare  the  computer- 
f cunilies .   These  qualities,  or  characteristics,  are  called 
selection  criteria. 

Criteria  that  are  used  to  evaluate  computer-families  are 
either  quantifiable  or  non-quantif ialDle .  The  qualities  that 
are  not  quantifiable  are  referred  to  as  qualitative  criteria 
and  are  evaluated  subjectively.  Those  characteristics  that 
are  quantifiable  and  measurable  by  an  established  standard 
are  called  quantitative  criteria,  and  are  evaluated 
objectively . 

Criteria  used  in  the  evaluation  process  will  be  placed 
at  different  levels  in  a  hierarchy.   At  the  top  of  the 
hierarchy  is  the  "Total  Evaluation  of  a  Computer-Family". 
The  second  level  consists  of  the  division  of  qualitative  and 
quantitative  criteria.  The  next  level  is  the  first  group  of 
selectible  criteria,  in  this  case  categories  of  computers. 


At  the  lowest  level  are  criteria  that  are  elements  of  a 
criterion  at  the  next  higher  level  in  the  hierarchy.   The 
criteria  at  each  level,  which  are  the  descriptors  of  a 
criterion  of  the  next  higher  level,  are  ranked  in  order  of 
importance  and  scaled  by  their  relative  weight.   This  is  the 
process  of  prioritizing. 

The  process  of  prioritization  will  be  conducted  using 
the  Analytical  Hierarchy  Process  developed  by  Saaty(1980). 
This  uses  a  one  to  one  comparison  of  all  the  items  at  the 
same  level,  producing  a  relative  ranking  and  weighting  of 
each  criterion  at  that  level  in  the  hierarchy.  These  are 
the  actions  that  take  place  during  Step  5  [Fig.  6] . 

Step  5.1.  Prioritize  the  overall  importance  of 
crualitative  versus  quantitative  criteria.   Because  there  are 
both  qualitative  and  quantitative  criteria  used  in  the 
evaluation  process,  prioritization  begins  with  the 
determination  of  the  weights,  or  percentage  of  the  total 
score,  for  each  of  these  groups  of  criteria.   This  is  a 
subjective  decision  made  without  the  use  of  the  AHP .   It  is 
the  second  level  in  the  hierarchy. 

Step  5.2.  Select  applicable  computer  categories.   Each 
category  is  evaluated  separately.   Prioritization  continues 
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with  the  selection  of  the  applicable  computer  categories  for 
the  computer-family  and  the  determination  of  the  relative 
weight  of  each  category.   Computer  category  is  the  third 
level  in  the  hierarchy.   The  categories  from  which  to  choose 
are : 

•  Mainframe  computer 

•  Super-minicomputer 

•  Minicomputer 

•  Super-microcomputer 

•  Microcomputer 

Step  5.3.  Select  criteria  for  each  computer  category. 
For  each  computer  category  designated  in  Step  5.2,  choose 
the  appropriate  main  categories  of  criteria  that  can  be  used 
in  the  evaluation  process.   The  main  categories  of  criteria, 
taken  from  the  list  in  Figure  2,    are: 

•  Hardware 

•  Software 

•  Communication 

•  Conversion 

•  Environment 

These  categories  of  criteria  are  the  fourth  level  in  the 
hierarchy . 


35 


step  5.4.  Select  sub-criteria  for  each  criterion  until 
there  are  no  more  sub-criteria.   For  each  main  category  of 
criteria.  Step  5.3,  in  each  main  category  of  computer,  Step 
5.2,    select  the  appropriate  s\ib-criteria.   If  a  criterion 
can  be  further  described  by  sub-criteria,  continue  the 
process  of  subdivision.   Subdivision  may  occur  down  three, 
four  or  even  five  more  levels. 

In  selecting  the  quantif iadDle  criteria,  care  must  be 
taken  to  select  relevant  items  and  to  not  continue 
subdividing  a  criterion  just  because  it  is  possible  to  do 
so.   The  criteria  selected  should  be  for  valid  items  of 
comparison  so  that  when  the  evaluation  of  the  computer- 
families  is  made,  and  there  is  a  difference  in  score  between 
computer-families,  the  difference  is  valid  and  meaningful. 

A  detailed  list  of  quantitative  criteria,  which  starts 
with  the  main  categories  of  criteria  at  Level  4  in  the 
hierarchy,  is  shown  in  Figure  8.   For  criterion  that  can  be 
subdivided,  the  sub-criteria  are  listed  in  hierarchy  Level 
5,  Level  6,  and  Level  7. 

The  qualitative  criteria,  even  though  they  are  evaluated 
much  more  subjectively,  can  also  be  prioritized  using 
Saaty's  procedures.   A  detailed  list  of  qualitative  criteria 
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that  may  be  used  in  comparison  of  computer-families  is  also 
shown  in  Figure  8 .   These  criteria  are  at  level  3  and  level 
4  in  the  qualitative  side  of  the  hierarchy.   The  hierarchy 
is  shown  in  Figure  7. 
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Level  4 
Hardware 


Memory 


Disks 


Diskette 
Readers 


Terminals 


Backup 


Number  of 

drives 
Capacity 


Size 

Number 

Density 


Keyboards 
Voice 


Color 
Graphics 
Laser 
Size 


Size 
Speed 


Capacity 

Expandc±>ility 

Speed 

Capacity 
ExpandcLbility 


Monochrome 
Color 

Graphics  abilities 
(CGA,EGA,VGA) 


Figure  8.1.  Detailed  List  of  Criteria,  Hardware 
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Level  4 


Level  6 


System 
software 


Operating  system  uniformity 

Utilities 

Compilers 

Compatibility 

Ability  to  modify 

Documentation 


Prograumning 
languages 


Compilers 
Interpreters 
Adherence  to  standards 
Options  and  extensions 


Development 
tools 


Additional 
software 


Application  generators 
Report  generators 
Full  screen  editors 
Debugging  aids 
Macro  facilities 


Word  processors 

Spreadsheets 

Database  Management  Systems 

E-Mail 


Software 
transferability 


Figure  8.2.   Detailed  List  of  Criteria,  Software 
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Level  4        Level  5 
Commun  i  c  a t  i  on 


Conversion 


Communication 
protocols 

Local  area 
networks 


Wide  area 
networks 


Integration 
of  current 
programs 

Integration 
of  data 
files 


Environment 


New  building 
requirements 

Special 
conditions 


Capacity  (Nxomber  of 

terminals) 
Maximum  distance 
Transmission  medixom 

Speed 


Transmission  medium 
Internet  capability 


Temperature 
Hximidity 


Figure  8.3.   Detailed  List  of  Criteria,  Communication, 
Conversion,  and  Environment. 
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Vendor 
Support 


Vendor' s 
reputation 


Manufacturer'  s 
reputation 


Training 
Maintenance 
Philosophy 
Practice 


User  opinion 

Trade  journal  evaluations 

User  opinion 

Trade  journal  evaluation 


Number  of  users 
of  computer-family 

Software  companies 
specializing  in 
this  computer-family 


Figure  8.4.   Detailed  List  of  Qualitative  Criteria 


Stap  5.5.  Prioritize  and  weight  all  categories, 
criteria  and  sub-criteria.   For  each  category  of  computer, 
the  appropriate  criteria  and  sub-criteria  are  listed.   After 
all  the  criteria  are  listed,  they,  along  with  the  computer 
categories  are  prioritized  and  weighted  using  Saaty's 
eigenvector  method.  [Fig.  9] 

As  described  earlier,  Saaty's  method  requires  a  pairwise 
comparison  of  those  criteria  that  are  located  at  the  same 
level  in  the  hierarchy.   Every  criterion  being  compared  is 
rated  against  every  other  criterion  in  the  saune  group.   A 
value  is  obtained,  based  on  the  values  from  Figure  5,  and 
inserted  into  the  matrix.   The  size  of  the  matrix  should  be 
limited  to  7  +/-  2  items  (Saaty,  1982) .   The  matrix 
generates  a  relative  weight  that  is  given  to  each  category 
or  criterion.   This  is  an  eigenvector;  it  has  a  decimal 
value  or  percentage  of  1.   The  total  value  of  all  the 
weights  generated  for  each  group  being  compared  is  1  or 
100%. 

B.   PRIORITIZING  AND  WEIGHTING:  A  DETAILED  EXAMPLE 

After  prioritizing  the  collective  qualitative  and 
quantitative   criteria.  Step  5.1,  and  then  selecting  the 
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applicable  computer  categories  and  appropriate  criteria  and 
Slab-criteria,  Steps  5.2  -  5.4,  a  decision  maker  is  ready  to 
begin  computing  priorities. 

In  this  example,  three  categories  of  computers,  selected 
in  Step  5.2,  are  placed  in  a  matrix  [Fig  9].   The  three 
categories  selected  are  Mainfrcune  computer,  Minicomputer, 
and  Microcomputer.    To  determine  the  relative  weight  for 
each  category,  a  decision  maker  has  to  compare  each  category 
to  the  other  two  categories,  one  at  a  time,  and  select  the 
number  1  to  9  which  best  represents  the  intensity  of 
importance  of  one  category  to  the  other.  The  scale,  shown  in 
full  in  Figure  5,  is  presented  here,  in  its  abbreviated 
form. 

Intensity  of 

Importance        Definition 

1  Equal  importance. 

3  Weak  importance  of  one  over  another. 

5  Essential  or  strong  importance. 

7  Demonstrated  importance. 

9  Absolute  importance. 

2,4,6,8         Intermediate  values  between  the  two 
adjacent  judgments. 
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The  numbers  that  are  inserted  into  the  matrix  compare 
the  item  on  the  left  with  the  item  in  the  top  row  and 
represent  the  value  from  the  scale  ahove .       If  the  item  on 
the  top  is  considered  more  important  than  the  item  on  the 
left,  a  fraction,  the  reciprocal  value  of  the  comparison,  is 
entered  into  the  matrix. 

Sample  values  have  been  entered  into  the  matrix. 
[Fig.  9.1]   Using  the  sample  values  an  eigenvector  will  be 
computed  for  each  computer  category.   The  eigenvector  will 
be  used  as  the  relative  weight  of  the  computer  category. 


1                        

1                 Hierarchy  Lovel 

3 

1 

1 
1 

1 

1   Computer  Category 

1   Main- 
1   frame 

Mini 

1 

Micro  1 

Mainframe 
1   Mini 
1   Micro 

1 
1    1/5 
•    1/3 

5 
1 
4 

3 
1/4   1 
1    1 

1 

Figure  9.1   The  Initial  Matrix 

The  matrix  is  then  normalized  by  adding  the  values  in 
each  column,  then  dividing  each  entry  in  each  column  by  the 
total  of  that  column,   A  new  matrix  with  normalized  values 
is  created.  [Fig.  9.2] 


I  Computer  Category 


Main 

Frame   Mini 


I   Mainframe 
I   Mini 
I   Micro 

I 

I  Column  total 


1 

5 

3 

1/5 

1 

1/4 

1/3 

4 

1 

1.533   10       4.25 


I  Computer  Category 


Main  I 

Frcune   Mini    Micro  I 


I  Mainfrcune 

I  Mini 

I  Micro 
I 


0.652      0.500      0.706    I 

0.131      0.100      0.059    I 

0.217       0.400       0.235     I 

I 


Figure  9.2  The  Normalized  Matrix 


The  last  step  is  to  average  over  the  rows  by  adding  the 
values  in  each  row  of  the  normalized  matrix  and  dividing  the 
rows  by  the  number  of  entries  in  each.   This  number  is  the 
eigenvalue  and  is  the  weight  that  is  given  to  that  category. 
[Fig.  9.3] 


Computed 

Relative 

Weight 

.652  +  .500  +  .706 

Mainframe  =    .6193 

3 


.131  +  .100  +  .059 

Mini  =    .0967 

3 

.217  +  .400  +  .235 

Micro  =    .2840 

3  

1.0000 


Figure  9.3   Computing  the  Eigenvalue 

The  total  relative  weight  for  all  computer  categories  is 
equal  to  1  or  100%. 

Consistency  of  the  values  must  be  taken  into  account  in 
computing  the  eigenvectors  using  the  calculated  consistency 
index  (CI) ,    dividing  the  CI  by  the  random  value  of 
consistency  from  a  table.   The  random  value  is  dependent  on 
the  number  of  items  in  the  matrix.   The  resultant  is  the 
consistency  ratio  (CR) .   The  CR  should  be  10%  or  less, 
indicating  judgments  in  comparing  items  in  the  matrix  were 
not  randomly  made.  (Saaty,  1982) 
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The  matrices  listed  in  Appendix  A  have  listed  in  them 
criteria  and  sub-criteria  from  hierarchy  Level  4  down 
through  Level  7.   For  this  example,  only  the  first  criterion 
from  the  previous  level  will  have  its  sub-criteria  listed  in 
the  next  lower  level.   The  numbers  along  the  top  row  of  each 
matrix  refer  to  the  corresponding  n\ambered  item  on  the  left 
side  of  the  matrix.   The  weight,  in  the  last  column,  is  the 
relative  weight,  the  eigenvalue,  of  the  criterion  naoned  in 
that  row,  in  the  left  column. 

For  this  example,  the  Mainframe  Computer  category  is 
used.   Its  relative  weight  has  already  been  calculated. 
[Fig. 9.3]   The  main  categories  of  criteria  are  selected,  and 
relative  weights  are  computed  for  each  of  these  criteria. 
The  first  matrix  determines  the  relative  weights  a  decision 
maker  places  on  each  of  the  main  categories  of  criteria  for 
each  of  the  computer  categories.   The  decisions  made  here 
have  an  effect  on  the  importance  of  the  decisions  made  lower 
in  the  hierarchy.   For  example,  if  hardware  turns  out  to  be 
worth  75%  in  the  evaluation  of  minicomputers,  the  remaining 
categories  of  criteria  are  worth  only  25%,  no  matter  how 
many  sub-criteria  are  used  in  the  evaluation  process.   This 
is  another  reason  not  to  subdivide  criteria  unnecessarily. 
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step  5.6.  Calculate  the  absolute  weights  for  all 
criteria  and  sub-criteria.   Following  the  method  proposed  by 
Borovits  and  Zviran  (1987) ,    a  scoring  system  that  makes  100 
points  the  maximum  score  attainable  for  a  computer-fcimily, 
will  be  used  in  conjunction  with  the  AHP . 

The  absolute  weight  for  a  criterion  is  computed  by 
multiplying  its  relative  weight  by  the  relative  weight  of 
each  of  its  predecessors  in  the  hierarchy  [Fig.  10],  or  by 
the  aibsolute  weight  of  its  immediate  predecessor.   When  a 
computer -family  is  evaluated  the  maximiim  score  available  for 
any  criterion  is  its  absolute  weight .   A  list  of  absolute 
weights  will  be  shown  on  an  evaluation  sheet,  Appendix  B 
used  in  conjunction  with  the  scoring  of  the  competing 
computer-families,  in  Step  7. 

An  example  of  how  an  absolute  weight  is  calculated  is 
shown  in  Figure  10.   A  sample  value  is  used  for  the  relative 
weight  of  quantitative  criteria.   The  relative  weight  for 
computer  categories,  mainframe  computer  was  computed  in 
Figure  9. 

The  process  is  demonstrated  in  Figure  10  and  is 
completed  for  each  criterion  in  the  hierarchy  of  a  computer- 
family.   The  absolute  weight  for  the  criteria  whose  relative 
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weights  were  calculated  in  Appendix  A  are  shown  on  the 
Vendor  Evaluation  Form  in  Appendix  B. 


Computer-       Quantitative         Mainfrsune 

Fcimily  Criteria  Computer 

Total  Weight    Relative  Weight      Relative  Weight 

=   1.00         =   0.80  =   0.6913 

Absolute  Weight  for  Mainframe  Computer  = 
(1.00)   X   (0.80)   X   (0.6913)  =   0.4954 


Figure  10.   Computing  an  absolute  weight  of  a 
criterion 
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VII.  THE  EVALUATION  PROCESS 

After  the  process  of  prioritizing  the  criteria  has  been 
completed,  the  process  of  evaluating  each  of  the  competing 
computer-families  begins.   In  Borovits  and  Zviran's  (1987) 
methodology  this  is  accomplished  as  part  of  Step  7. 

In  response  to  the  request  for  proposal  (RFP)  that  was 
sent  out  to  vendors,  bids  for  proposed  computer-families 
will  have  been  received.   These  bids  then  have  to  be 
analyzed  and  evaluated  with  scores  assigned.   In  Steps  8,  9, 
and  10,  the  highest  rated  computer-families,  on  the  basis  of 
score,  are  selected  for  further  evaluation  and  testing  and, 
ultimately,  one  will  be  selected  as  the  winner. 

The  scope  of  Step  7  is  the  initial  evaluation  and 
scoring  of  the  proposed  computer-families,  submitted  in 
response  to  the  RFP.   As  stated  previously,  this  is 
completed  by  two  s\ab-steps. 

Step  7.1.  Assign  each  relevant  model  from  a  proposed 
computer-family  to  a  category,  according  to  predetermined 
criteria .   In  Step  5.2,  computer  categories  were  selected. 
The  decision  maker  chose  the  computer  categories  that  met  an 


organization's  needs.   Each  relevant  computer  model  from  the 
proposed  family  will  be  placed  in  the  appropriate  computer 
category  and  will  be  evaluated  by  the  criteria  already 
selected  for  that  category. 

The  determination  as  to  which  category  a  computer  will 
be  placed  is  based  on,  but  not  limited  to,  such  factors  as 
CPU,  memory  or  disk  drives.   The  decision  as  to  what  factors 
will  constitute  placement  into  a  particular  category  will 
have  been  determined  when  the  categories  were  selected  in 
Step  5.2. 

Step  7.2.  Evaluate  each  computer-family  in  accordance 
with  the  criteria  established  in  Step  5.   Using  the 
evaluation  form.  Appendix  B,  each  computer-family  is 
evaluated,  criterion  by  criterion,  with  the  result  posted  on 
the  form.   The  total  for  the  computer-family  is  then 
computed  and  a  final  score  is  established. 

The  maximum  score  possible  for  any  criterion  is  its 
c±)solute  weight.   The  score  assigned  and  posted  on  the  form 
is  on  a  scale  of  0-1.   Complying  with  the  criterion 
completely  is  scored  as  1,  anything  less  is  a  decimal  value 
less  than  1 .   The  posted  score  is  multiplied  by  the  aibsolute 
weight  to  yield  the  cibsolute  score  for  that  criterion.   All 
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of  the  absolute  scores  are  added  to  yield  the  final  score 
for  the  computer-family.   Only  the  lowest  level  criteria  are 
scored.   The  maximum  total  score  is  100.   When  this  process 
has  been  completed  for  each  of  the  computer-families,  an 
objective  comparison  based  on  the  final  score  may  be  made. 
The  three  or  four  highest  rated  computer- fauni lies  may  then 
be  evaluated  further  by  operational  and  benchmark  testing. 

In  Appendix  B,  the  relative  weights  computed  in  Appendix 
A  have  been  inserted  into  the  evaluation  sheet.   Using  those 
relative  weights,  the  absolute  weights  for  all  criteria 
cited  in  the  example  have  been  computed. 
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VIII.  CONCLUSION 

The  process  of  selecting  a  computer-family  is  a  complex 
procedure.   The  goal  for  a  decision  maker,  responsible  for 
selecting  a  computer-family,  is  to  select  the  correct  line 
of  products  for  an  organization.   Because  of  the  complexity 
of  the  selection  process,  a  formalized  methodology  makes  the 
process  more  objective. 

Using  the  methodology  presented  here,  along  with 
methodology  developed  by  Borovits  and  Zviran  (1987) ,  an 
objective  selection  process  is  created.   The  process  allows 
the  designation  of  appropriate  criteria  to  be  used  in  the 
selection  of  a  computer-family.   The  appropriate  criteria 
are  those  selected  by  an  organization  trying  to  select  a 
computer-family,  based  on  an  evaluation  of  its  needs. 

Once  criteria  have  been  selected,  they  are  objectively 
prioritized  and  weighted,  establishing  their  net  value  and 
absolute  weight  in  an  overall  evaluation  of  a  computer- 
fcimily.   Each  computer- famiily  is  evaluated  and  scored 
separately,  but  in  accordance  with  the  prioritized  and 
weighted  criteria. 
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In  spite  of  the  careful  process  of  prioritizing  and 
weighting,  total  objectivity  can  not  be  achieved. 
Individuals  comparing  criteria  will  still  be  making 
subjective  judgments  as  to  the  relative  importance  of  one 
criterion  versus  another.   This  cannot  be  avoided. 

Subjectivity  in  the  comparison  of  criteria  may  be 
counteracted  by  checking  for  consistency.   The  ability  to 
measure  the  consistency  in  the  comparison  of  criteria  adds  a 
check  and  balance  to  the  process  not  previously  present. 

By  following  the  procedures  developed  in  this  thesis, 
the  process  of  selecting  a  computer-f aunily  is  made  reliable 
and  objective.   The  end  product  of  this  process,  a  computer- 
family  that  best  meets  the  needs  of  an  organization,  may  be 
chosen  with  the  knowledge  that  the  correct  computer-family 
was  selected. 
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APPENDIX  A 

Sample  values,  using  the  guidelines  from  Figure  5,  have 
been  inserted  into  the  matrices.  Using  the  process  in  Step 
5.5,  each  matrix  was  normalized.   The  relative  weight  listed 
in  the  last  column  is  the  eigenvalue  of  the  criterion  in  the 
first  column  in  that  same  row. 


] 

Computer  Category 

[.evel  3 

1 

2 

3 

Relative 
Weight 

1. Mainframe 
2. Mini 
3. Micro 

1 
1/5 
1/3 

5 

1 
4 

3 

1/4 

1 

0.6193 
0.0964 
0.2842 

1.0000 

] 

Main  Categories 

of  Criteria  (Mainframe) 

^evel 

1 

4 
2 

3 

4 

5 

Relative 
Weight 

1. Hardware 

2. Software 

3 . Communication 

4 .Conversion 

5. Environment 

1 

2 
1/5 
1/3 
1/6 

1/2 

1 
1/5 
1/3 
1/7 

5 
5 
1 
4 
1/3 

3 
3 
1/4 
1 
1/4 

6 

7 
3 

4 

1 

0.3024 
0.4119 
0.0775 
0.1656 
0.0426 

1.0000 

56 


Level 

5 

Relative 

Hardware 

1    2 

3 

4 

5 

6 

Weight 

1. Memory 

1    3 

6 

2 

5 

4 

0.3633 

2. Disks 

1/3   1 

3 

1/3 

3 

5 

0.1663 

3. Diskette  Readers 

1/6  1/3 

1 

1/5 

1/3 

1/3 

0.0420 

4. Terminals 

1/2   3 

5 

1 

3 

5 

0.2631 

5. Printers 

1/5  1/3 

3 

1/3 

1 

3 

0.0986 

6 . Backup 

1/4  1/5 

3 

1/5 

1/3 

1 

0.0667 

Level  6 

Memory                   1      1        2 

Relative 
Weight 

l.Main 
2. RAM 

1        1/3 
3         1 

0.25 
0.75 

1.00 

] 
Main  Memory 

^evel  7 

12      3 

Relative 
Weight 

1. Capacity 
2.Expandibility 
3. Speed 

RAM 

1  1/2    1/3 

2  1     1/3 

3  3      1 

1         2 

0.1593 
0.2518 
0.5889 

1.0000 

Relative 
Weight 

1. Capacity 
2.Expandibility 

1        1/4 
4         1 

0.2 
0.8 

APPENDIX  B 


Vendor  Evaluation  Sheet 


Relativa 
Weight 


Abaoluta 
Weight 


Absolut* 
Scora     Score 


Total 

1.000 

1.000 

Quantitative 

.8000 

.8000 

Mainfrcune 

.6193 

.4954 

Hardware 

.3024 

.1498 

Memory 

.3633 

.0544 

Main 

.2500 

.0136 

Capacity 

.1593 

.0022 

Expandibility 

.2518 

.0034 

Speed 

.5889 

.0080 

RAM 

.7500 

.0408 

Capacity 

.2000 

.0082 

Expandibility 

.8000 

.0326 

Disks 

.1663 

.0249 

Number  of  drives 

Capacity 

Diskette  Readers 

.0420 

.0063 

Size 

Number 

Density 
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Vendor   Evaluation   Sheet  Relative  Absolute 

Mainframe   Hardware    (cont'd)         weight    weight 


Absolute 
Score  Score 


Terminals 

.2631 

.0394 

Monitors 

Monochrome 

Color 

Graphics  abilities 

Keyboards 

Voice 

Printers 

.0986 

.0148 

Color 

Graphics 

Laser 

Size 

Backup 

.0667 

.0010 

Size 

Speed 

Software 

.4119 

.2041 

System  software 

Op.  system  uniformity 

Utilities 

Compilers 

Compatibility 

Ability  to  modify 

Documentation 
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Vendor   Evaluation    Sheet  Relative  Absolute 

Mainframe    Software    (cont'd)         weight     weight 


Absolute 
Score  Score 


Programming  languages 

Compilers 

Interpreters 

Adherence  to  standards 

Options  and  extensions 

Development  tools 

Application  generators 

Report  generators 

Full  screen  editors 

Debugging  aids 

Macro  facilities 

Additional  software 

Word  processors 

Spreadsheets 

Database  mgmt .  systems 

E-Mail 

Software  transferability 
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ior  Evaluation  Sheet             Relative  Absolute 

Mainframe  (cont'd)            weight  weight 

Score 

Absolute 
Score 

Communication 

.0775 

.0384 

Communication  protocols 

Local  area  networks 

Capacity  (#  of  terminals) 

Maximxam  distance 

Transmission  medium 

Speed 

Wide  area  networks 

Transmission  medium 

Internet  capadDility 

Conversion 

.1656 

.0821 

Integrate  current  progms . 

Integration  of  data  files 

Environment 

.0426 

.0211 

New  building  requirements 

Special  Conditions 

Temperature 

Humidity 
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